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INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-2884 


November  12,  1999 

MEMORANDUM  FOR  DIRECTOR,  DEFENSE  LOGISTICS  AGENCY 

SUBJECT:  Audit  Report  on  Defense  Logistics  Agency  Logistics  Year  2000  End-to- 
End  Test  Planning  (Report  No.  D-2000-036) 


We  are  providing  this  report  for  your  information  and  use.  This  report  is  one  of 
a  series  of  reports  being  issued  by  the  Inspector  General,  DoD,  in  accordance  with  an 
informal  partnership  with  the  DoD  Chief  Information  Officer  to  identify  progress  made 
by  DoD  Components  that  are  preparing  information  and  technology  systems  for 
year  2000  compliance.  We  considered  management  comments  on  a  draft  of  this  report 
in  preparing  the  final  report. 

Comments  from  the  Defense  Logistics  Agency  conformed  to  the  requirements  of 
DoD  Directive  7650.3  and  left  no  unresolved  issues.  Therefore,  no  additional 
comments  are  required. 


We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Tilghman  Schraden  at  (703)  604-9186  (DSN  664-9186) 
(tschraden@dodig.osd.mil)  or  Ms.  Kathryn  Palmer  at  (703)  604-8840  (DSN  664-8840) 
(kpalmer@dodig.osd.mil).  See  Appendix  E  for  the  report  distribution.  The  audit  team 
members  are  listed  inside  the  back  cover. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Report  No.  D-2000-036  November  12, 1999 

(Project  No.  9LD-9024.04) 

Defense  Logistics  Agency  Logistics  Year  2000  End-to-End 

Test  Planning 


Executive  Summary 


Introduction.  This  is  one  in  a  series  of  reports  being  issued  by  the  Inspector  General, 
DoD,  in  accordance  with  an  informal  partnership  with  the  Chief  Information  Officer, 
DoD,  to  monitor  DoD  efforts  to  address  the  year  2000  computing  challenge.  For  a 
complete  listing  of  audit  projects  addressing  the  issue,  see  the  year  2000  web  pages  on 
the  IGnet  at  http://www.ignet.gov. 

The  DoD  Year  2000  Management  Plan  (DoD  Management  Plan)  assigns  responsibility 
to  the  Principal  Staff  Assistants  for  ensuring  the  end-to-end  functional  process  flows 
that  support  their  functional  area  are  assessed  either  in  a  Joint  Staff  or  commander  in 
chief  year  2000  operational  evaluation,  a  Service-sponsored  system  integration  test,  or  a 
functional  area  year  2000  end-to-end  test.  The  Principal  Staff  Assistants  are  also 
responsible  for  planning,  executing,  and  evaluating  all  mission-critical  systems  not 
otherwise  tested  and  ensuring  that  processes  that  fall  within  their  purview  are  evaluated. 
The  Deputy  Under  Secretary  of  Defense  (Logistics)  (DUSD[L])  acts  on  behalf  of  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology,  the  Principal  Staff 
Assistant  for  logistics,  in  performing  those  functions  for  the  logistics  functional  area. 
Logistics  end-to-end  test  planning  was  accomplished  through  the  “Logistics  Capstone 
Operational  Assessment  Plan  for  Year  2000”  (Logistics  Capstone  Plan). 

Logistics  functional  end-to-end  testing  was  divided  into  three  phases.  Level  I  was  intra- 
Component  testing,  and  Level  II  was  inter-Component  testing.  Level  III  testing  was  to 
be  conducted  as  required  to  perform  retesting.  The  DUSD(L)  provided  oversight  for 
Level  II  testing  while  delegating  responsibility  for  Level  I  testing  to  the  Components. 
Level  II  testing  began  May  25,  1999,  and  was  completed  July  14,  1999.  The  Joint 
Interoperability  Test  Command  concluded  in  the  final  report,  “Logistics  Year  2000 
End-to-End  Level  II  Exercise  Evaluation  Report,”  October  1999,  that  the  core  logistics 
processes  will  continue  unaffected  by  year  2000-related  issues  and  that  the  logistics 
automated  information  systems  will  operate  as  a  whole  to  support  the  five  core  logistics 
processes.  The  Defense  Logistics  Agency  (DLA)  completed  Level  I  testing  on 
April  21,  1999.  DLA  test  results  indicated  that  there  were  no  year  2000-related 
failures  during  Level  I  testing  and  that  the  test  objectives  were  met.  DUSD(L) 
representatives  stated  that  Level  III  testing  would  not  be  required  because  of  the 
successful  demonstration  of  year  2000  capabilities  by  the  logistics  systems  participating 
in  the  test  of  the  five  core  logistics  processes. 
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Objective.  The  audit  objective  was  to  evaluate  the  effectiveness  of  the  year  2000  end- 
to-end  tests  planned  for  the  logistics  functional  area.  This  report,  the  fourth  in  a  series 
on  logistics  end-to-end  testing,  addresses  the  overall  end-to-end  test  planning 
accomplished  by  DLA. 

Results.  DLA  end-to-end  test  planning  for  core  logistics  processes  generally  met  the 
requirements  outlined  in  the  DoD  Management  Plan  and  the  Logistics  Capstone  Plan. 

In  response  to  the  practical  limitations  imposed  by  resource  constraints  and  calendar 
time  remaining,  DLA  and  the  Services,  in  conjunction  with  the  Logistics  Year  2000 
Interface  Assessment  Working  Group  and  the  DUSD(L),  prioritized  the  logistics 
processes  and  data  flows  to  be  included  in  testing  based  on  criticality  to  the  warfighter. 
They  identified  five  critical  core  logistics  processes  for  inter-Component  testing.  DLA 
participated  in  the  testing  of  all  five  of  those  processes.  DLA  identified  seven  core 
processes  for  intra-Component  testing.  DLA  included  9  of  its  26  mission-critical 
logistics  systems  in  the  logistics  functional  end-to-end  testing.  Two  additional  systems 
were  included  in  higher  level  testing  during  commander  in  chief  operational  evaluations 
and  one  system  was  tested  in  the  environmental  security  functional  end-to-end  testing. 
An  additional  system  did  not  require  higher  level  testing.  DLA  exercised  contingency 
plans  for  all  nine  mission-critical  systems  included  in  the  logistics  end-to-end  testing. 
However,  DLA  did  not  conduct  higher  level  testing  for  13  mission-critical  logistics 
systems  as  required  by  the  DoD  Management  Plan.  DLA  also  did  not  document  risk 
assessments  for  its  core  logistics  processes  and  systems.  As  a  result,  DLA  needs  to 
document  risk  assessments  and  develop  a  risk  mitigation  strategy  that  will  ensure  that 
its  critical  processes  and  systems  will  perform  the  operational  mission  in  the  year  2000. 
See  the  Finding  section  for  details. 

Management  Actions  to  Mitigate  Risk.  During  the  course  of  the  audit,  DLA 
provided  the  audit  team  with  information  on  actions  taken  and  planned  to  further 
mitigate  risk  of  year  2000-related  failures.  DLA  initiated  a  two-stage  code'  scanning 
program.  In  addition,  the  Chief  Information  Officer,  DLA,  conducted  a  formal  risk 
mitigation  meeting  in  September  1999.  The  purpose  of  that  meeting  was  to  review 
post-remediation  experiences  and  activities  of  other  organizations  and  to  develop  a  plan 
of  action  that  focuses  on  addressing  those  areas  within  the  overall  DLA  mission-critical 
information  technology  infrastructure.  DLA  also  initiated  a  tracking  system  to  monitor 
higher  level  testing  of  DLA  mission-critical  systems. 

Summary  of  Recommendation.  We  recommend  that  the  Chief  Information  Officer, 
DLA,  develop  a  risk  management  plan  for  inclusion  in  the  DUSD(L)  plan  that  includes 
a  risk  assessment  and  mitigation  strategy  for  the  core  logistics  processes,  with  special 
emphasis  on  those  processes  and  systems  that  were  not  included  in  higher  level  testing. 

Management  Comments.  DLA  concurred  with  the  recommendation,  stating  that  DLA 
is  working  with  the  Logistics  Capstone  operational  test  coordinator  who  has  been  tasked 
to  complete  the  DUDS(L)  risk  assessment  plan  and  that  DLA  will  ensure  that  its  core 
logistics  processes  and  systems  are  included  in  the  risk  mitigation  strategy.  A 
discussion  of  management  comments  is  in  the  Finding  section  of  the  report,  and  the 
complete  text  is  in  die  Management  Comments  section. 
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Defense  Logistics  Agency 


Background 


Executive  Order.  Because  of  the  potential  failure  of  computers  to  function 
throughout  the  Government,  the  President  issued  Executive  Order  13073,  “Year 
2000  Conversion,”  February  4,  1998,  making  it  policy  that  Federal  agencies 
ensure  that  no  critical  Federal  program  experiences  disruption  because  of  the 
year  2000  (Y2K)  problem.  The  order  requires  that  the  head  of  each  agency 
ensure  that  efforts  to  address  the  Y2K  problem  receive  the  highest  priority 
attention  in  the  agency. 

Public  Law.  Public  Law  105-261,  “National  Defense  Authorization  Act  for 
Fiscal  Year  1999,”  October  17,  1998,  Section  334(b),  directs  that  the  Secretary 
of  Defense  ensure  that  “all  mission  critical  systems  that  are  expected  to  be  used 
if  the  Armed  Forces  are  involved  in  a  conflict  in  a  major  theater  of  war  are 
tested  in  at  least  two  exercises.”  In  addition,  Section  334(d)  states: 

“Alternative  Testing  Method.  In  the  case  of  an  information  technology  or 
national  security  system  for  which  a  simulated  year  2000  test  as  part  of  a 
military  exercise  described  in  subsection  (c)  is  not  feasible  or  presents  undue 
risk,  the  Secretary  of  Defense  shall  test  the  system  using  a  functional  end-to-end 
test  or  through  a  Defense  Major  Range  and  Test  Facility  Base.” 

DoD  Y2K  Management  Strategy.  In  his  role  as  the  DoD  Chief  Information 
Officer,  the  Senior  Civilian  Official,  Office  of  the  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications,  and  Intelligence),  issued  the 
“DoD  Year  2000  Management  Plan,  Version  2.0”  (DoD  Management  Plan)  in 
December  1998.  The  DoD  Management  Plan  required  DoD  Components  to 
implement  a  five-phase  (awareness,  assessment,  renovation,  validation,  and 
implementation)  Y2K  management  process  to  be  completed'by  December  31, 
1998,  for  mission-critical  systems. 

The  DoD  Management  Plan  also  provides  guidance  for  implementing  the 
Deputy  Secretary  of  Defense  memorandum,  “Year  2000  (Y2K)  Verification  of 
National  Security  Capabilities,”  August  24,  1998,  that  requires  that  each 
Principal  Staff  Assistant  (PSA)  of  the  Office  of  the  Secretary  of  Defense  “verify 
that  all  functions  under  his  or  her  purview  will  continue  unaffected  by  Y2K 
issues.  ”  That  verification  was  to  be  performed  after  completion  of  the 
five-phase  management  approach  that  culminated  with  completion  of  the 
implementation  phase,  December  31,  1998.  That  further  testing,  to  be 
conducted  during  the  first  half  of  1999,  was  planned  and  conducted  from  a 
mission  perspective  rather  than  a  system  perspective  and  would  increase  the 
confidence  that  any  errors  or  omissions  in  system  remediation  would  be  found. 
The  Deputy  Under  Secretary  of  Defense  (Logistics)  (DUSD[LJ)  acts  on  behalf 
of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology,  the  PSA  for 
logistics. 

DoD  Logistics  End-to-End  Planning.  The  DUSD(L)  implemented  and 
executed  key  components  of  the  DoD  Management  Plan  in  his  efforts  to 
adequately  plan  for  and  manage  logistics  functional  end-to-end  testing.  Test 
planning  was  accomplished  through  the  “Logistics  Capstone  Operational 
Assessment  Plan  for  Year  2000”  (Logistics  Capstone  Plan),  dated  October  30, 
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1998,  and  approved  in  November  1998.  The  Logistics  Capstone  Plan  provided 
the  overall  strategy  for  conduct  of  the  logistics  end-to-end  testing  and  was 
coordinated  with  the  Services,  the  Defense  Logistics  Agency  (DLA),  the  Joint 
Interoperability  Test  Command,  and  the  Joint  Staff.  The  October  1998 
Logistics  Capstone  Plan  was  updated  in  February  1999  and  again  in  May  1999 
to  reflect  evolving  schedules  and  processes.  Its  name  was  changed  to  “Logistics 
Capstone  Plan  for  Year  2000  End-to-End  Test”  as  part  of  the  February  update. 
In  this  report,  unless  otherwise  noted,  Logistics  Capstone  Plan  refers  to  the 
May  20,  1999,  version. 

The  Logistics  Capstone  Plan  defines  three  levels  of  testing  and  delegates 
responsibility  for  each.  The  multilevel  test  approach  consisted  of  intra- 
Component  events  (Level  I),  inter-Component  events  (Level  II),  and  post-test 
activities  that  include  retest  (Level  III).  Level  I  tests  were  designed  to  ensure 
processes  and  systems  within  a  Component’s  organizational  boundaries  are  Y2K 
ready.  Level  II  testing  was  to  verify  core  processes  and  information  flows  that 
involve  more  than  a  single  Component  are  Y2K  ready.  The  execution  and 
oversight  of  the  Level  I  testing  was  delegated  to  the  Components  (the  Services 
and  DLA),  while  DUSD(L)  focused  on  the  Level  II  testing  and  post-test  events, 
such  as  retest,  during  Level  III.  Independent  validation  and  verification  for  test 
planning,  execution,  and  reporting  of  Level  I  and  II  testing  for  DLA  systems 
was  achieved  through  the  use  of  an  independent  contractor.  Independent 
validation  and  verification  of  Level  II  testing  was  also  achieved  through  the  use 
of  the  Joint  Interoperability  Test  Command  for  test  planning,  execution,  and 
reporting. 


Objective 


The  audit  objective  was  to  evaluate  the  effectiveness  of  the  Y2K  end-to-end  tests 
planned  for  the  logistics  functional  area.  This  report,  the  fourth  in  a  series  on 
logistics  end-to-end  testing,  addresses  the  overall  end-to-end  test  planning 
accomplished  by  DLA.  See  Appendix  A  for  a  discussion  of  the  audit  scope  and 
methodology  and  Appendix  B  for  a  summary  of  prior  coverage. 
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Defense  Logistics  Agency  Planning  for 
Logistics  Functional  End-to-End  Testing 

DLA  end-to-end  test  planning  for  core  logistics  processes  generally  met 
the  requirements  outlined  in  the  DoD  Management  Plan  and  the  Logistics 
Capstone  Plan.  In  response  to  the  practical  limitations  imposed  by 
resource  constraints  and  calendar  time  remaining,  DLA  and  the  Services, 
in  conjunction  with  the  Logistics  Y2K  Interface  Assessment  Working 
Group  (IAWG)1  and  the  DUSD(L),  prioritized  the  logistics  processes 
and  data  flows  to  be  included  in  testing  based  on  criticality  to  the 
warfighter.  They  identified  five  critical  core  logistics  processes  for 
inter-Component  testing.  DLA  participated  in  the  testing  of  all  five  of 
those  processes.  DLA  identified  seven  core  processes  for  intra- 
Component  testing.  DLA  included  9  of  its  26  mission-critical  logistics 
systems  in  the  logistics  functional  end-to-end  testing.  Two  additional 
systems  were  included  in  higher  level  testing  during  commander  in  chief 
operational  evaluations.  One  system  was  tested  in  the  environmental 
security  end-to-end  test.  One  system  did  not  require  testing.  DLA 
exercised  contingency  plans  for  all  nine  mission-critical  systems  included 
in  the  logistics  end-to-end  testing.  However,  DLA  did  not  conduct 
higher  level  testing  for  its  remaining  13  mission-critical  logistics 
systems.  DLA  did  not  document  risk  assessments  for  its  core  logistics 
processes  and  systems.  As  a  result,  DLA  needs  to  document  risk 
assessments  and  develop  a  risk  mitigation  strategy  that  will  ensure  that 
its  critical  processes  and  systems  will  perform  the  operational  mission  in 
the  year  2000. 


End-to-End  Test  Guidance 


The  DoD  Management  Plan,  Appendix  I,  provides  guidance  for  the  planning 
and  implementation  of  functional  end-to-end  testing.  It  assigns  responsibility  to 
the  PSAs  for  “ensuring  the  end-to-end  functional  process  flows  that  support 
their  functional  area  are  assessed  either  in  a  JS/CINC  [Joint  Staff/Commander  in 
Chief]  Y2K  Op  Eval  [Operational  Evaluation],  a  Service-sponsored  System 
Integration  Test,  or  through  a  Functional-Area  Y2K  End-to-End  Test.” 
Appendix  I  also  states  that  the  PSAs’  responsibilities  include  “planning, 
executing,  and  evaluating  all  mission-critical  systems  not  otherwise  tested  and 
for  ensuring  that  processes  that  fall  within  their  purview  are  evaluated.”  The 
Logistics  Capstone  Plan  provided  the  overall  strategy  for  conduct  of  the  DoD 
logistics  end-to-end  testing. 


'The  Logistics  Y2K  IAWG  membership  was  composed  of  DoD  Component  representatives  and  was 
chaired  by  the  Director,  Logistics  Systems  Modernization. 
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DLA  Planning  for  End-to-End  Testing 


The  DLA  end-to-end  test  planning  for  core  logistics  processes  generally  met  the 
requirements  outlined  in  the  DoD  Management  Plan  and  the  Logistics  Capstone 
Plan.  DLA  implemented  the  end-to-end  testing  guidance  provided  in  the 
Logistics  Capstone  Plan  with  the  issuance  of  the  “Year  2000  (Y2K)  DLA 
Capstone  Testing  Level  I  and  Level  II  DLA  DID-Software  Test  Plan  (STP), 
DI-IPSC-81438,  Version  5.0,”  undated  (DLA  Capstone  Test  Plan).  The  DLA 
Capstone  Test  Plan  defines  the  DLA  strategy  for  its  participation  in  the  logistics 
end-to-end  testing  for  Level  I  and  Level  II  testing.  The  overall  objective  of 
DLA  participation  in  the  DoD  logistics  end-to-end  test  effort  was  to  determine 
whether  DLA  mission-critical  systems  could  interface  correctly  both  internally 
with  other  DLA  systems  (Level  I  testing)  and  externally  with  other  DoD 
systems  (Level  II  testing)  in  a  Y2K  environment.  As  required  by  the  Logistics 
Capstone  Plan,  the  DLA  Capstone  Test  Plan  addressed  areas  such  as  end-to-end 
test  strategy,  critical  core  processes,  mission-critical  systems  that  supported  the 
core  processes,  and  test  limitations. 

DLA  started  Level  I  end-to-end  testing  of  its  core  processes  and  systems  on 
April  5,  1999,  and  completed  the  testing  on  April  21,  1999.  DLA  started  Level 
II  end-to-end  testing  of  its  core  processes  and  systems  on  May  25,  1999,  and 
completed  the  testing  on  July  14,  1999.  The  DoD  Management  Plan  calls  for 
final  test  reports  to  be  completed  within  30  days  of  completion  of  testing.  In  the 
final  test  report,  “Logistics  Year  2000  End-to-End  Level  II  Exercise  Evaluation 
Report,”  October  1999,  the  Joint  Interoperability  Test  Command  concluded  that 
the  core  logistics  processes  will  continue  unaffected  by  Y2K-related  issues  and 
that  the  logistics  automated  information  systems  will  operate  as  a  whole  to 
support  the  five  core  logistics  processes  included  in  Level  II  testing.  Level  II 
testing  identified  two  anomalies  that  were  not  Y2K-related,  one  that  was  date- 
related  and  one  that  was  not,  for  two  of  the  six  DLA  systems  tested.  DLA  had 
completed  corrective  actions  for  those  systems  by  the  end  of  Level  II  testing. 
The  anomalies  in  the  two  DLA  systems,  the  Distribution  Standard  System  and 
the  Standard  Automated  Materiel  Management  System,  were  corrected  in  June 
and  July,  respectively.  DLA  results  from  the  Level  I  testing  indicated  no  Y2K- 
related  failures  and  that  the  objectives  of  the  test  were  met. 

DLA  Test  Responsibilities.  The  DLA  Capstone  Test  Plan  assigns 
responsibility  to  the  DLA  Y2K  Test  Director  and  the  DLA  Systems  Design 
Center  (DSDC)2  for  conducting  qualification  testing  of  the  mission-critical 
logistics  systems  and  core  processes.  Qualification  testing  includes  the  formal 
documentation  of  test  scenarios,  documentation  of  trouble  reports,  and 
certification  of  the  applicable  programs.  DSDC  personnel  in  Columbus,  Ohio; 
New  Cumberland  and  Philadelphia,  Pennsylvania;  and  Ogden,  Utah,  provided 
support  for  the  systems  that  were  included  in  the  Level  I  and  II  testing.  DSDC 
was  also  responsible  for  performing  Level  I  and  II  testing  of  core  processes  and 


2The  DSDC  was  disestablished  in  November  1998  and  its  operations  and  personnel  transitioned  to  other 
elements  within  DLA.  For  simplicity,  we  have  referred  to  the  role  played  by  the  former  DSDC 
personnel  by  their  former  organizational  name  throughout  this  report. 
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for  performing  configuration  management  control  of  the  applications  during  the 
testing.  Systems  analysts,  computer  programmers,  and  computer  specialists 
performed  the  Level  I  and  II  testing.  DLA  Level  I  testing  was  verified  and 
validated  by  both  DSDC  personnel  and  an  independent  contractor.  Independent 
verification  and  validation  of  Level  II  testing  was  achieved  through  the  use  of 
the  Joint  Interoperability  Test  Command  in  support  of  the  DUSD(L). 

Core  Processes.  DLA  and  the  Services,  in  conjunction  with  the  IAWG  and  the 
DUSD(L),  agreed  that  all  mission-critical  systems  and  processes  could  not  be 
assessed  during  the  logistics  functional  Level  II  end-to-end  testing  because  of 
time  and  resource  constraints.  As  a  result,  they  identified  8  out  of  15  core 
supply  and  materiel  management  processes  as  mission-critical.  The  Logistics 
Capstone  Plan  defines  those  mission-critical  processes3  as  being  “so  dependent 
on  automation,  that  within  hours  or  days  of  an  automation  system  being  needed 
and  not  available,  a  warfighting  mission  is  impaired.”  The  eight  mission-critical 
processes  were  farther  refined  to  five  core  processes  determined  to  be  required 
to  the  support  the  warfighter.  Those  five  core  processes  were  the  basis  for  the 
logistics  functional  end-to-end  testing  conducted  during  the  Level  II  (inter- 
Component)  testing.  The  five  core  processes  were  requisition,  shipment, 
receipt,  inventory,  and  asset  status.  In  addition,  DLA  identified  seven  core 
processes  for  Level  I  (intra-Component)  testing.  The  seven  intra-Component 
processes  included  four  of  the  processes  from  Level  II  (requisition,  shipment, 
receipt,  and  inventory)  and  three  additional  processes  (contract  award,  contract 
shipment,  and  shipment  alert). 

Systems  Supporting  Core  Processes.  The  general  approach  taken  by  DLA  and 
the  Services  was  to  identify  critical  functional  processes  and  then  identify  the 
systems  that  supported  those  processes.  DLA  identified  six  mission-critical 
logistics  systems  that  supported  the  Level  II  testing  of  the  five  core  processes. 
DLA  identified  nine  mission-critical  logistics  systems  that  supported  Level  I 
testing  of  the  seven  DLA  processes.  All  of  the  Level  II  systems  were  also  tested 
during  the  Level  I  testing.  Appendix  C  contains  a  list  of  the  systems  included  in 
Level  I  and  Level  II  testing  and  the  logistics  processes  that  they  supported. 

Test  Limitations.  Because  all  logistics  processes  and  mission-critical  system 
interfaces  could  not  be  tested  within  the  time  available,  DLA  limited  its  testing 
in  several  areas.  Specifically,  test  limitations  occurred  in  the  test  environment, 
date  crossings  tested,  and  transactions  tested. 

Test  Environment.  The  DLA  Capstone  Test  Plan  stated  that  the 
purpose  of  Level  I  and  Level  II  end-to-end  testing  was  to  ensure  interoperability 
in  Y2K  environments  of  mission-critical  system  interfaces.  Further,  testing  was 


’To  avoid  confusing  mission-critical  process  and  mission-critical  systems,  the  term  core  processes  is  used 
throughout  this  report  to  refer  to  the  five  processes  identified  for  end-to-end  testing. 
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to  include  all  files,  interface  control  documents,  and  support  utilities  needed  to 
validate  the  Logistics  Capstone  Plan.  Specifically,  Level  I  and  Level  II  end-to- 
end  testing  ensured  that: 

•  Y2K  platforms  met  or  exceeded  the  performance  of  the  current 
operating  environments  without  change  to  the  system  functionality, 

•  all  program  support  utilities  functioned  properly  in  the  new  Y2K 
environment, 

•  uploads  and  downloads  of  data  functioned  properly,  and 

•  applications  would  function  and  perform  in  the  Y2K  environments 
using  the  crossover  dates. 

The  limitations  in  the  DLA  test  environment  were  as  follows. 

•  System  testing  did  not  validate  the  support  utility  programs. 

•  Tests  were  not  conducted  in  production  environments4  but  used 
representative  test  environments. 

•  Testing  was  not  performed  to  ensure  complete  integration  of  modules 
and  hardware  and  to  ensure  that  all  increments’  functionalities  were 
present  and  operable. 

•  Testing  was  not  an  uninterrupted  end-to-end  test.  Because  the  test 
environment  could  not  be  configured  to  simulate  all  systems  at  one 
time,  the  test  was  accomplished  systematically  by  business  area  and 
process  cycle.  ■ 

•  Qualification  testing  was  limited  to  a  representative  sample  of  critical 
DLA  applications. 

Although  not  listed  as  a  test  limitation  in  the  DLA  Capstone  Test  Plan,  the 
Logistics  Capstone  Plan  acknowledged  that  not  all  DLA  commodity  groups 
could  be  tested  because  of  limited  calendar  time  and  availability  of  test 
environments.  Specifically,  out  of  six  DLA  commodity  groups  managed  by  the 
Standard  Automated  Materiel  Management  System,  one  (electronics)  was 
included  in  the  inter-Component  Level  II  testing  and  one  (construction)  was 
included  in  the  intra-Component  Level  I  testing.  The  Standard  Automated 
Materiel  Management  System  is  used  by  three  DLA  Supply  Centers  to  manage 
inventory  for  six  commodities:  construction,  electronics,  clothing  and  textile, 
medical,  industrial,  and  general  supplies.  Because  each  commodity  executes 
unique  programs  within  die  Standard  Automated  Materiel  Management  System, 
there  are  six  copies,  or  versions,  of  the  Standard  Automated  Materiel 
Management  System.  Each  copy  supports  a  commodity.  Commodities  not 


4Production  environments  are  the  environments  in  which  software  applications  operate  on  a  day-to-day 
basis 
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included  in  either  the  Level  I  or  II  testing  were  clothing  and  textile,  industrial, 
medical,  and  general  supplies.  The  medical  supplies  commodity,  which  also 
uses  the’ Defense  Medical  Logistics  Supply  System,  was  included  in  the 
functional  end-to-end  testing  being  conducted  by  the  PSA  for  health  affairs. 
The  subsistence  commodity  was  included  under  Level  I  testing  for  the  Defense 
Integrated  Subsistence  Management  System,  but  was  not  included  in  Level  II 
testing.3 


Date  Crossings  Tested.  Date  scenarios  tested  in  Level  II  testing  were 
fiscal  year  (September  30,  1999,  to  October  1,  1999),  calendar  year 
(December  31,  1999,  to  January  1,  2000),  and  leap  day  (February  28,  2000,  to 
February  29,  2000,  and  February  29,  2000,  to  March  1,  2000).  The  DLA 
Capstone  Test  Plan  stated  that  because  of  technical  constraints  on  the  DLA  test 
bed  and  the  time  available  for  implementing  Y2K  compliant  systems,  DLA 
selected  a  group  of  Y2K  dates  for  evaluation  to  ensure  that  the  systems  operate 
to  maximum  capability  and  that  system  functionality  remains  intact.  Date 
scenarios  tested  in  Level  I  testing  were  fiscal  year,  calendar  year,  and  the 
quantitative  location  reconciliation  process  date  of  January  9,  2000.  The 
quantitative  location  reconciliation  process  date  is  the  date  for  a  DLA  internal 
inventory  management  report  on  which  the  inventory  balances  of  the  inventory 
control  points  are  aligned  with  the  balances  at  the  storage  activity.  The  leap  day 
date  crossings  were  not  tested  in  Level  I  testing  because  of  time  constraints  on 
accomplishing  the  testing.  A  baseline  test  was  performed  to  compare  current 
data  with  the  test  results. 

Transactions  Tested.  DLA  limited  the  number  and  type  of  transactions 
it  tested  in  Level  I  and  Level  II  end-to-end  testing.  During  the  process  of 
refining  the  Logistics  Capstone  Plan,  the  IAWG  selected  for  end-to-end  testing 
supply  transactions  for  electronics  items.  The  DLA  Capstone  Test  Plan 
scheduled  308  test  cases  for  Level  I  testing.  The  test  cases  identified  the 
purpose  of  each  test,  such  as  to  test  the  receipt  of  inventory,  and  identified  the 
systems  performing  the  test,  the  national  stock  numbers  that  would  be  processed 
through  the  systems  as  part  of  the  testing,  and  the  expected  test  results.  Those 
test  cases  were  developed  for  each  system  participating  in  the  Level  I  testing  and 
supported  the  seven  processes  that  DLA  identified  for  Level  I  testing.  Level  I 
testing  was  performed  to  ensure  that  processes  and  systems  within  DLA  would 
function  properly  across  the  fiscal  year,  the  calendar  year,  and  in  the  year  2000. 

DLA  processed  176  transactions  during  Level  II  testing.  They  included  24 
Army,  82  Navy,  39  Air  Force,  and  31  Marine  Corps  transactions  that 
represented  supply  requisitions  between  Services.  The  Level  II  transactions 
included  28  DLA-managed  national  stock  numbers. 


5DLA  manages  the  subsistence  commodity  group  but  does  not  use  the  Standard  Automated  Materiel 
Management  System  to  do  so. 
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Higher  Level  Testing  of  Mission-Critical  Systems 

DLA  included  12  of  its  26  mission-critical  logistics  systems  in  one  of  two  types 
of  higher  level  tests,  functional  area  end-to-end  testing  and  commander  in  chief 
operational  evaluations,  that  are  described  in  the  DoD  Management  Plan.  DLA 
did  not  initiate  any  Service  integration  testing,  a  third  type  of  higher  level 
testing  described  in  the  DoD  Management  Plan.  DLA  did  not  conduct  any 
higher  level  testing  for  14  mission-critical  logistics  systems. 

Requirement  for  Higher  Level  Testing.  In  keeping  with  Public  Law  105-261, 
Section  334(b),  the  DoD  Management  Plan  requires  that  all  mission-critical 
systems  that  would  be  directly  involved  in  a  major  theater  war  be  tested  in  a 
commander  in  chief  operational  evaluation.  Also,  each  of  those  systems  must 
be  tested  a  second  time,  in  an  additional  commander  in  chief  operational 
evaluation,  a  functional  area  end-to-end  test,  or  a  Service-sponsored  system 
integration  test.  All  other  mission-critical  systems  must  be  tested  at  least  once 
in  either  a  functional  area  end-to-end  test  or  a  Service-sponsored  system 
integration  test. 

Systems  Tested.  Of  the  26  mission-critical  logistics  systems  listed  in  the  DoD 
Y2K  Reporting  Database,  DLA  identified  a  total  of  9  systems  that  were  to  be 
tested  in  Level  I  intra-Component  testing.  Six  of  the  nine  systems  were  also 
included  in  Level  II  inter-Component  testing.  Two  additional  DLA  systems 
were  included  or  scheduled  to  be  included  in  at  least  two  commander  in  chief 
operational  evaluations.  Three  systems  that  were  included  in  commander  in 
chief  operational  evaluations  were  also  included  in  Level  I  and  Level  II  testing 
and,  as  a  result,  met  the  requirement  for  two  higher  level  tests.  One  system,  the 
Defense  Reutilization  and  Marketing  Automated  Information  System,  was  tested 
in  the  environmental  security  end-to-end  testing.  The  remaining  14  DLA 
mission-critical  logistics  systems  were  not  included  in  higher  level  testing  as 
defined  by  the  DoD  Management  Plan.  Appendix  D  contains  a  list  of  the  DLA 
mission-critical  systems  and  their  higher  level  test  status. 

Systems  Not  Tested.  DLA  cited  three  reasons  for  not  including  the  14  mission- 
critical  systems  in  higher  level  testing:  12  systems  did  not  exchange  date 
information  with  other  systems;  the  deployed  portion  of  one  system  operated  in 
a  stand-alone  environment;  and  one  system  did  not  support  the  core  processes 
identified  for  functional  end-to-end  testing.  However,  only  one  system,  the 
deployed  portion  of  a  system  operating  in  a  stand-alone  environment,  met  the 
criteria  for  exclusion  from  higher  level  testing  provided  by  the  DoD 
Management  Plan.  DLA  did  not  include  13  systems  in  higher  level  testing  as 
required  by  the  DoD  Management  Plan. 

DLA  identified  12  systems  that  did  not  exchange  dates  with  other  systems  and, 
therefore,  DLA  did  not  require  testing  of  those  systems.  The  DoD  Management 
Plan  states  that  inclusion  in  one  of  the  higher  level  testing  events  is  not  explicitly 
required  for  systems  that  have  no  date  dependency  or  that  are  in  a  stand-alone 
environment.  The  DoD  Management  Plan  also  allows  waivers  for  systems  that 
cannot  be  included  in  a  functional  test.  However,  DLA  did  not  report  any  of 
those  12  systems  as  not  having  date  dependency  or  as  having  a  waiver.  The 
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DoD  Y2K  Reporting  Database  indicated  that  the  12  systems  were  certified  as 
Y2K  compliant  as  a  result  of  Y2K  remediation  efforts  that  involved  independent 
testing  or  an  independent  audit  of  the  system,  and  that  testing  was  completed 
using  a  2-  or  4-digit  year  format.  In  addition,  the  certification  level  assigned  to 
those  systems  did  not  indicate  that  the  systems  did  not  process  date-related  data. 
As  a  result,  those  12  systems  appear  to  be  date  dependent  and  should  have  been 
included  in  higher  level  testing  designed  to  test  interfaces. 

DLA  excluded  the  Fuels  Automated  System  (Base  Level)  from  higher  level 
testing  because  that  portion  of  the  system  does  not  interface  with  other  systems 
in  the  configuration  currently  deployed.  According  to  the  DoD  Management 
Plan,  higher  level  testing  was  not  required  because  the  deployed  portion  of  the 
system  was  in  a  stand-alone  environment.  Although  the  Fuels  Automated 
System  is  an  integrated  system  consisting  of  several  modules,  DLA  reported  the 
Fuels  Automated  System  (Base  Level)  as  a  mission-critical  system  in  the  DoD 
Y2K  Reporting  Database.  The  Enterprise  Level  portion  of  the  system  is  under 
development  and  will  interface  with  external  financial  and  procurement  systems. 
DLA  explained  that  the  development  of  the  Enterprise  Level  portion  would  not 
be  completed  until  after  January  2000. 

DLA  reported  that  1  of  the  14  mission-critical  systems  not  included  in  a  higher 
level  test,  the  Federal  Logistics  Information  System,  did  not  support  the  core 
processes  identified  for  Level  I  and  Level  II  testing.  Support  of  a  core  process 
was  not  one  of  the  criteria  for  determining  the  requirement  for  higher  level 
testing.  Higher  level  testing  is  still  required  for  the  system  because  it  is  defined 
as  a  mission-critical  system  having  a  date  dependency.  The  DoD  Management 
Plan  states  that  inclusion  in  one  of  the  higher  level  testing  events  is  not  explicitly 
required  for  systems  that  have  no  date  dependency  or  that  are  in  a  stand-alone 
environment.  The  DoD  Management  Plan  also  allows  waivers  for  systems  that 
cannot  be  included  in  a  functional  test.  However,  DLA  did  not  report  the 
system  as  not  having  date  dependency,  as  operating  in  a  stand-alone 
environment,  or  as  having  a  waiver. 


Contingency  Plan  Testing 


DLA  exercised  the  contingency  plans  for  eight  of  the  nine  mission-critical 
logistics  systems  included  in  Level  I  and  II  logistics  end-to-end  testing  by  the 
June  30,  1999,  cutoff  date  required  by  the  DoD  Management  Plan.  The 
contingency  plan  for  the  remaining  system  was  exercised  during 
September  1999.  The  DoD  Y2K  Reporting  Database  showed  that  DLA  had 
completed  contingency  plans  for  all  nine  mission-critical  logistics  systems 
included  in  the  DLA  Level  I  and  II  logistics  end-to-end  testing. 

The  Logistics  Capstone  Plan  requires  that  all  thin-line  systems  supporting  the 
identified  core  processes  have  an  effective  contingency  plan.  In  addition,  the 
Logistics  Capstone  Plan  states  that  the  contingency  plans  must  be  developed  and 
validated  by  operators,  must  be  resourced,  and  must  be  tested.  Further,  the 
Logistics  Capstone  Plan  states  that  contingency  test  plans  should  address  the  test 
objectives,  test  approach,  required  equipment  and  resources,  necessary 
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personnel,  schedules  and  locations,  test  procedures,  expected  results,  and  exit 
criteria.  The  DoD  Management  Plan  established  a  target  completion  date  of 
March  31,  1999,  for  operational  contingency  plans  and  June  30,  1999,  for 
exercising  those  contingency  plans.  However,  the  Logistics  Capstone  Plan 
extended  the  target  completion  date  for  testing  of  individual  contingency  plans  to 
September  1,  1999.  As  of  October  14,  1999,  the  contingency  plans  for  the  nine 
mission-critical  DLA  logistics  systems  had  been  exercised. 


Measures  to  Minimize  Risk  of  Y2K-Related  System  Failures 

DLA  did  not  document  the  risk  assessments  performed  during  the  process  of 
prioritizing  logistics  processes  for  inclusion  in  end-to-end  testing  as  required  by 
the  DoD  Management  Plan  and  the  Logistics  Capstone  Plan.  The  DoD 
Management  Plan  states  that  the  Y2K  event  master  planning  sessions  (the 
Logistics  IAWG  meetings)  were  to  identify  and  prioritize  core  processes  and 
perform  risk  assessments.  The  Logistics  Capstone  Plan  made  each  Service  and 
agency  responsible  for  identifying  the  risks  that  affect  their  portions  of  the 
Level  I  and  II  end-to-end  functional  tests  and  their  specific  systems  tests  and  for 
reporting  on  risk  management  and  mitigation  efforts.  The  Logistics  Capstone 
Plan  identified  four  general  categories  of  corporate-level  risk:  funding; 
scheduling;  scope  of  testing;  and  test  environment.  It  also  assigned  each 
category  a  risk  rating  of  high,  medium,  or  low,  based  on  probability  of 
occurrence  and  consequences  of  occurrence,  as  well  as  listed  the  mitigation  of  a 
particular  risk.  The  Logistics  Capstone  Plan  stated  that  the  discussion  of 
corporate-level  risks  was  an  initial  risk  assessment.  In  addition,  the  Logistics 
Capstone  Plan  stated  that  a  complete  risk  mitigation  plan  and  detailed 
procedures  for  managing  and  reporting  on  risks  will  be  incorporated  in  an 
overall  risk  management  plan. 

DUSD(L)  had  planned  to  complete  a  risk  management  plan  on  all  core  logistics 
processes  by  September  1999.  We  determined  that  the  DLA  Capstone  Test  Plan 
did  not  include  guidance  on  documenting  risk  assessments  or  preparing  or 
submitting  a  risk  analysis  or  mitigation  plan  to  the  DUSD(L)  or  to  DLA 
management  for  the  DLA  processes  and  systems.  As  a  result,  as  of  August  30, 
1999,  DLA  had  not  documented  risk  assessments  or  completed  a  risk 
management  plan  that  documented  the  DLA  risk  mitigation  strategy  for  review 
and  inclusion  in  the  overall  DUSD(L)  risk  management  plan.  Therefore,  the 
DUSD(L)  did  not  have  sufficient  information  to  complete  a  risk  management 
plan  for  core  logistics  processes  by  September  1999,  and  may  not  be  able  to 
meet  the  revised  goal  of  November  1999. 


Management  Actions  to  Mitigate  Risk 


During  the  course  of  the  audit,  DLA  provided  the  audit  team  with  information 
on  actions  taken  and  planned  to  further  mitigate  risk  of  Y2K-related  failures. 
DLA  initiated  a  two-stage  code  scanning  program.  The  first  stage  consists  of  a 
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representative  sample  scan  of  each  mission-critical  system’s  code  to  detect 
Y2K-related  errors.  The  second  stage  consists  of  scanning  100  percent  of  a  . 
system’s  code. 

In  addition,  the  Chief  Information  Officer,  DLA,  conducted  a  formal  risk 
mitigation  meeting  in  September  1999.  The  purpose  of  that  meeting  was  to 
review  post-remediation  experiences  and  activities  of  other  organizations  and  to 
develop  a  plan  of  action  that  focuses  on  those  areas  within  the  overall  DLA 
mission-critical  information  technology  infrastructure  where  the  most  significant 
degree  of  continuity  or  mission  fulfillment  risk  remains.  DLA  also  initiated  a 
tracking  system  to  monitor  higher  level  testing  of  DLA  mission-critical  systems. 


Summary 


DLA  generally  complied  with  the  DoD  Management  Plan  and  the  Logistics 
Capstone  Plan  in  planning  and  managing  its  efforts  under  Level  I  and  Level  II 
logistics  end-to-end  testing.  DLA  participated  in  the  testing  of  the  five  core 
logistics  processes  identified  for  Level  II  testing.  In  addition,  DLA  identified 
seven  core  processes  for  Level  I  testing,  which  included  four  of  the  processes 
from  Level  II  and  three  additional  processes.  DLA  included  12  of  26  mission- 
critical  systems  in  higher  level  testing.  One  system  did  not  require  testing. 
However,  DLA  did  not  ensure  that  the  other  13  mission-critical  systems  that 
were  not  included  in  end-to-end  testing  or  commander  in  chief  operational 
evaluations  were  included  in  the  higher  level  tests  required  by  the  DoD 
Management  Plan. 

DLA  met  the  DoD  Management  Plan  requirements  for  the  exercising  of 
contingency  plans  but  did  not  complete  this  task  until  after  the  June  30,  1999, 
date  required  by  the  DoD  Management  Plan.  The  contingency  plan  for  one  of 
the  nine  mission-critical  logistics  systems  included  in  Level  I  and  Level  II 
testing  was  not  exercised  until  September  1999.  In  addition,  DLA  did  not 
document  the  risk  assessments  performed  during  the  process  of  prioritizing 
logistics  processes  for  inclusion  in  end-to-end  testing  as  required  by  the  DoD 
Management  Plan  and  the  Logistics  Capstone  Plan. 


Management  Comments  on  the  Finding  and  Audit  Response 


DLA  Comments.  DLA  partially  agreed  with  the  finding,  pointing  out  that  12 
of  the  DLA  mission-critical  logistics  systems  not  tested  did  not  require  higher 
level  testing  because  they  did  not  exchange  date-related  information.  DLA  also 
stated  that  the  Defense  Reutilization  and  Marketing  Automated  Information 
System  was  tested  in  an  environmental  security  end-to-end  test;  the  Fuels 
Automated  System  (Base  Level)  was  a  stand-alone  system;  and  the  Federal 
Logistics  Information  System  was  not  identified  by  the  commanders  in  chief  or 
the  PSAs  as  part  of  a  critical  logistics  process  requiring  testing.  DLA  also 
stated  that  it  had  tested  all  mission-critical  systems  at  the  Component  level. 
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DLA  provided  an  update  to  the  status  of  contingency  plan  testing,  stating  that 
the  ninth  critical  system  that  supported  the  core  processes  included  in  the  end-to- 
end  testing  had  been  exercised  in  September  1999.  In  addition,  DLA  pointed 
out  that  it  had  conducted  and  documented  a  comprehensive  risk  assessment  of  all 
business  procedures,  and  that  the  audit  team’s  request  for  risk  assessment 
documentation  was  confined  to  end-to-end  test  planning. 

Audit  Response.  We  disagree  with  the  DLA  statement  that  12  systems  did  not 
require  testing  because  they  did  not  exchange  date-related  information  with 
other  systems.  The  12  systems  cited  by  DLA  are  date-dependent  systems  that 
require  higher  level  testing  in  accordance  with  guidance  provided  by  the  DoD 
Management  Plan.  We  have  also  revised  the  report  based  on  the  DLA 
explanation  of  the  status  of  the  Fuels  Automated  System.  We  added  information 
noting  that  the  Defense  Reutilization  and  Marketing  Automated  Information 
System  was  included  in  an  environmental  security  end-to:end  test.  We  disagree 
with  the  DLA  statement  that  the  Federal  Logistics  Information  System  did  not 
require  higher  level  testing  because  it  was  not  identified  by  the  commander  in 
chiefs  or  the  PSAs  as  part  of  a  critical  logistics  process.  The  DoD  Management 
Plan  requires  that  all  mission-critical  systems  having  date  dependency  be  tested 
at  least  once  in  a  higher  level  test.  The  Federal  Logistics  Information  System 
was  identified  by  DLA  as  a  mission-critical  system  in  the  DoD  Y2K  Reporting 
Database.  We  revised  the  number  of  systems  not  tested  because  they  did  not 
support  core  processes  identified  for  Level  I  and  II  testing  based  on  the  DLA 
identification  of  the  Federal  Logistics  Information  System  as  being  the  one 
system  in  that  category.  In  addition,  we  revised  the  reported  number  of  systems 
not  tested  and  the  status  of  contingency  plans  exercised.  This  audit  report 
addresses  the  logistics  functional  end-to-end  testing  as  noted  by  DLA,  and  does 
not  address  risk  assessments  for  internal  DLA  business  processes. 


Recommendation  and  Management  Comments 

We  recommend  that  the  Chief  Information  Officer,  Defense  Logistics 
Agency,  develop  a  risk  management  plan  for  inclusion  in  the  Deputy  Under 
Secretary  of  Defense  (Logistics)  plan  that  includes  a  risk  assessment  and 
mitigation  strategy  for  the  core  logistics  processes,  with  special  emphasis  on 
those  processes  and  systems  that  were  not  included  in  higher  level  testing. 

DLA  Comments.  DLA  concurred,  stating  that  DLA  is  working  with  the 
Logistics  Capstone  operational  test  coordinator  who  has  been  tasked  to  complete 
the  DUSD(L)  risk  assessment  plan.  The  mitigation  actions  that  result  from  the 
DUSD(L)  risk  assessment  will  be  worked  within  the  Logistics  IAWG.  As  an 
IAWG  member,  DLA  will  ensure  that  its  core  logistics  processes  and  systems 
are  included  in  the  risk  mitigation  strategy. 


Appendix  A.  Audit  Process 


This  is  one  in  a  series  of  reports  being  issued  by  the  Inspector  General,  DoD,  in 
accordance  with  an  informal  partnership  with  the  Chief  Information  Officer, 
DoD,  to  monitor  DoD  efforts  to  address  the  Y2K  computing  challenge.  For  a 
listing  of  audit  projects  addressing  the  issue,  see  the  Y2K  web  pages  on  IGnet  at 
http://www.ignet.gov. 


Scope  and  Methodology 

Work  Performed.  We  reviewed  the  Y2K  test  planning  efforts  of  DLA  for  the 
logistics  functional  end-to-end  testing.  We  evaluated  the  Y2K  planning  efforts 
of  DLA  and  compared  those  efforts  with  the  criteria  contained  in  the  DoD 
Management  Plan  and  the  Logistics  Capstone  Plan.  We  reviewed  Public  Law 
105-261,  Section  334;  the  Deputy  Secretary  of  Defense  memorandum  of 
August  24,  1998;  the  DoD  Management  Plan;  the  Logistics  Capstone  Plan;  the 
DLA  Capstone  Test  Plan;  and  other  guidance  regarding  the  testing  of  mission- 
critical  logistics  systems.  Documents  reviewed  were  dated  from  October  1998 
through  September  1999.  We  interviewed  personnel  within  the  offices  of  the 
DUSD(L)  and  DLA.  We  also  interviewed  contractor  representatives  involved 
with  end-to-end  testing. 

DoD-Wide  Corporate-Level  Goals.  In  response  to  the  Government 
Performance  and  Results  Act,  DoD  established  2  DoD-wide  corporate-level 
goals  and  7  subordinate  performance  goals.  This  report  pertains  to  achievement 
of  the  following  goal  (and  subordinate  performance  goal). 

Goal:  Prepare  now  for  an  uncertain  future  by  pursuing  a  focused 
modernization  effort  that  maintains  U.S.  qualitative  superiority  in  key 
warfighting  capabilities.  Transform  the  force  by  exploiting  the 
Revolution  in  Military  Affairs  and  reengineer  the  Department  to  achieve 
a  21st  century  infrastructure.  Performance  Goal:  Transform  U.S. 
military  forces  for  the  future.  (00-DoD-2.2) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  objectives  and  goals  in  the 
Information  Technology  Management  Functional  Area. 

•  Objective:  Become  a  mission  partner.  Goal:  Serve  mission 
information  users  as  customers.  (ITM-1.2) 

•  Objective:  Provide  services  that  satisfy  customer  information  needs. 
Goal:  Modernize  and  integrate  Defense  information  infrastructure. 
(ITM-2.2) 
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•  Objective:  Provide  services  that  satisfy  customer  information  needs. 
Goal:  Upgrade  technology  base.  (ITM-2.3) 

High-Risk  Area.  In  its  identification  of  risk  areas,  the  General  Accounting 
Office  has  specifically  designated  risk  in  resolution  of  the  Y2K  problem  as  high. 
This  report  provides  coverage  of  that  problem  and  of  the  overall  Information 
Management  and  Technology  high-risk  area. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  program  audit  from 
July  through  September  1999  in  accordance  with  auditing  standards  issued  by 
the  Comptroller  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD.  We  did  not  use  computer-processed  data  for  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 

Management  Control  Program.  We  did  not  review  the  management  control 
program  related  to  the  overall  audit  objective  because  DoD  recognized  the  Y2K 
issue  as  a  material  management  control  weakness  area  in  the  FY  1998  Annual 
Statement  of  Assurance. 
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Appendix  B.  Summary  of  Prior  Coverage 


The  General  Accounting  Office  and  the  Inspector  General,  DoD,  have  conducted 
multiple  reviews  related  to  Y2K  issues.  General  Accounting  Office  reports  can  be 
accessed  over  the  Internet  at  http://www.gao.gov.  Inspector  General,  DoD,  reports 
can  be  accessed  over  the  Internet  at  http://www.dodig.osd.mil.  The  reports  most 
relevant  to  the  subject  matter  of  this  report  are  listed  below. 


General  Accounting  Office 


General  Accounting  Office  Report  No.  GAO/AIMD-99-172  (OSD  Case  No.  1823), 
“Defense  Computers:  Management  Controls  Are  Critical  to  Effective  Year  2000 
Testing,”  June  30,  1999. 


Inspector  General,  DoD 


Inspector  General,  DoD,  Report  No.  D-2000-033,  “Army  Logistics  Year  2000  End-to- 
End  Test  Planning,”  November  5,  1999. 

Inspector  General,  DoD,  Report  No.  00-021,  “Air  Force  Logistics  Year  2000  End-to- 
End  Test  Planning,”  October  26,  1999. 

Inspector  General,  DoD,  Report  No.  00-002,  “Year  2000  End-to-End  Testing: 
Logistics  Capstone  Plan,”  October  1,  1999. 
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Appendix  C.  Defense  Logistics  Agency 

Mission-Critical  Logistics  Systems  and 
Processes  -  Level  I  and  Level  II  Testing 


Mission-Critical  Processes 


Asset 

Contract 

Contract 

Shipment 

System 

Requisition 

Shipment 

Receipt 

Inventory 

Control 

Award 

Shipment 

Alert 

DESEX 

I 

I 

I 

I 

I 

I 

DISMS 

I 

I 

I 

I 

DMARS 

I, II 

I, II 

I, II 

I,II 

II 

DNCS 

I, II 

I,II 

1,11 

I,II 

II 

DSS 

I,  II 

i,n 

I,  II 

I,  II 

I 

I 

DSS  Bridge 

i,  n 

i,  ii 

I,  II 

I,  II 

EDIPAS 

i 

MADS 

i,  ii 

i,  ii 

I, II 

I,  II 

I 

I 

I 

SAMMS 

i,  ii 

i,  ii 

I 

i,n 

I 

I 

DESEX  Defense  Supply  Expert  System 

DISMS  Defense  Integrated  Subsistence  Management  System 

DMARS  Defense  Automatic  Addressing  System  Micro  Automated  Routing  System 

DNCS  Defense  Automatic  Addressing  System  Center  (DAASC)  Network  Control  System 

DSS  Distribution  Standard  System 

DSS  Bridge  Distribution  Standard  System  Bridge 

EDIPAS  Electronic  Data  Interchange  Pass-through 

MADS  Messaging  Accountability  Delivery  System 

SAMMS  Standard  Automated  Materiel  Management  System 


Appendix  D.  Defense  Logistics  Agency  - 

Higher  Level  Test  Status  of 
Mission-Critical  Logistics  Systems 


System1 

Level  I 
Test 

Level  II 
Test 

CINC2 

Test 

Not 

Tested 

DAASACP 

X3 

DAISY 

X4 

DAMES 

X3 

DARS 

X3 

DEPRA 

X3 

DESEX 

X 

DFAMS 

X 

DIELOG 

X3 

DISMS 

X 

DMARS 

X 

X 

X 

DMRS 

X3 

DNCS 

X 

X 

X 

DODAAD 

X3 

DSS 

X 

X 

DSS  Bridge 

X 

X 

EDIPAS 

X 

FAS 

Xs 

FLIS 

Xs 

ILCS 

X3 

JTAV 

X 

MADS 

X 

X 

MAPAD 

X3 

MIDTIER 

X3 

MSOS 

X3 

SAMMS 

X 

X 

X 

SPLC 

X3 

'System  acronyms  are  defined  on  the  following  page. 

’Commander  in  chief. 

’System  identified  by  DLA  as  not  requiring  higher  level  testing  because  it  does  not  exchange  date 
information  with  other  systems. 

"DAISY  was  included  in  the  environmental  security  end-to-end  testing,  not  the  logistics  end-to-end 
testing. 

’Deployed  portion  of  FAS  operates  in  a  stand-alone  environment  and  higher  level  testing  is  not  required. 

’System  identified  by  DLA  as  not  requiring  higher  level  testing  because  it  was  not  identified  by 
commanders  in  chief  or  PSAs  as  part  of  a  critical  core  logistics  process. 


System  Acronym 

System  Name 

DAASACP 

Defense  Automatic  Addressing  System  (DAAS)  Allied 
Communications  Procedures 

DAISY 

Defense  Reutilization  and  Marketing  Automated  Information  System 

DAMES 

DAAS  Center  Automated  Message  Exchange  System 

DARS 

DAAS  AUTODIN  Replacement  System 

DEPRA 

Defense  Program  for  Redistribution  of  Assets 

DESEX 

Defense  Supply  Expert  System 

DFAMS 

Defense  Fuels  Automatic  Management  System 

DIELOG 

DMSC  Integrated  Email  Logistics 

DISMS 

Defense  Integrated  Subsistence  Management  System 

DMARS 

DAAS  Micro  Automated  Routing  System 

DMRS 

DAAS  Master  Routing  System 

DNCS 

DAAS  Center  Network  Control  System 

DODAAD 

DoD  Activity  Address  Directory 

DSS 

Distribution  Standard  System 

DSS  Bridge 

Distribution  Standard  System  Bridge 

EDIPAS 

Electronic  Data  Interchange  Pass-through 

FAS 

Fuels  Automated  System  (Base  Level) 

FLIS 

Federal  Logistics  Information  System 

ILCS 

International  Logistics  Communication  System 

JTAV 

Joint  Total  Asset  Visibility 

MADS 

Messaging  Accountability  Delivery  System 

MAPAD 

Military  Assistance  Program  Address  Directory 

MIDTIER 

Mid-Tier  Server 

MSOS 

Master  Source  of  Supply 

SAMMS 

Standard  Automated  Materiel  Management  System 

SPLC 

Standard  Point  Location  Project 
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Appendix  E.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Deputy  Under  Secretary  of  Defense  (Logistics  and  Materiel  Readiness) 

Director,  Logistics  System  Modernization 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Deputy  Chief  Information  Officer  and  Deputy  Assistant  Secretary  of  Defense  (Chief 
Information  Officer  Policy  and  Implementation) 

Principal  Director  for  Year  2000 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 
Inspector  General,  Department  of  the  Army 

Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Navy 
Inspector  General,  Department  of  the  Navy 
Inspector  General,  Marine  Corps 

Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 
Inspector  General,  Department  of  the  Air  Force 

Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 

Chief  Information  Officer,  Defense  Contract  Audit  Agency 
Director,  Defense  Information  Systems  Agency 

Inspector  General,  Defense  Information  Systems  Agency 
Chief  Information  Officer,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 

Chief  Information  Officer,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Office  of  Information  and  Regulatory  Affairs 

National  Security  Division  Special  Projects  Branch 
Federal  Chief  Information  Officers  Council 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 

Director,  Defense  Information  and  Financial  Management  Systems,  Accounting  and 
Information  Management  Division 
Inspector  General,  General  Services  Administration 

Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Special  Committee  on  the  Year  2000  Technology  Problem 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Government  Reform 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology,  Committee  on  Science 
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Defense  Logistics  Agency  Comments 


DEFENSE  LOGISTICS  AGENCY 
HEADQUARTERS 

8725  JOHN  J.  KINGMAN  ROAD,  SUPTE  2533 
FT.  BELYOIR,  VIRGINIA  22060-6221 


IN  REPLY 

REFER  TO 


OCT  1  4  ms 


memorandum  for  assistant  inspector  general  for  auditing, 

DEPARTMENT  OF  DEFENSE 


SUBJECT:  DoD  IG  Final  Report,  Defense  Logistics  Agency  Logistics  Year  2000  End- 
to-End  Test  Planning  (Project  No.  9LD-9024.04) 

This  is  in  response  to  the  September  30, 1999,  Draft  Report.  DLA  partially  concurs  with 
the  finding  in  the  draft  audit  report  that  "DLA  did  not  conduct  higher  level  testing  for  its 
remaining  15  mission-critical  logistics  systems."  Although  the  statement  is  essentially 
correct,  12  of  those  systems  do  not  require  higher  level  testing  because  they  do  not 
exchange  date  information.  The  draft  report  also  states  that  "DLA  did  not  document  risk 
assessments  for  its  core  logistics  processes  and  systems."  DLA  has  conducted  and 
documented  a  comprehensive  risk  assessment  of  all  of  its  business  processes.  See 
attached  comments. 


RAYMOND  A.  ARCHER  III 
Rear  Admiral,  SC,  USN 
Deputy  Director 

Attachments 
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Final  Report 
Reference 


T 


OCT  14  TO 


Revised 


Revised 


SUBJECT:  Defense  Logistics  Agency  Logistics  Year  2000  End-to-End  Test 
Planning  (Project  No.  9LD-9024.04) 

FINDING:  Defense  Logistics  Agency  Planning  for  Logistics  Functional  End-to- 
End  Testing.  DLA  end-to-end  test  planning  for  mission-critical  logistics 
processes  generally  met  the  requirements  outlined  in  the  DoD  Management  Plan 
and  the  Logistics  Capstone  Plan.  In  response  to  the  practical  limitations  imposed 
by  resource  constraints  and  calendar  time  remaining,  DLA  and  the  Services,  in 
conjunction  with  the  Logistics  Y2K  Interface  Assessment  Working  Group 
(IA WG)  and  the  DUSD(LX  prioritized  the  logistics  processes  and  data  flows  to  be 
included  in  testing  based  on  criticality  to  the  warfighter.  They  identified  five 
critical  core  logistics  processes  for  inter-Componeht  testing.  DLA  participated  in 
the  testing  of  all  five  of  those  processes  DLA  identified  seven  mission-critical 
processes  for  intra-Component  testing.  DLA  included  9  of  its  26  mission-critical 
logistics  systems  in  the  logistics  functional  end-to-end  testing.  Two  additional 
system s  were  included  in  higher  level  testing  during  commander  in  chief 
operational  evaluations.  However,  DLA  did  not  conduct  higher  level  testing  for 
its  remaining  15  mission-critical  logistics  systems.  Of  the  nine  mission-critical 
systems  included  in  the  logistics  end-to-end  testing,  contingency  plans  for  eight 
systems  had  been  exercised  DLA  did  not  document  risk  assessments  for  its  core 
logistics  processes  and  systems  As  a  result,  DLA  needs  to  document  risk 
assessments  and  develop  a  risk  mitigation  strategy  that  will  ensure  that  its  critical 
processes  and  systems  will  perform  the  operational  mission  in  the  year  2000. 

DLA  COMMENTS:  Partially  concur. 


The  statement  that  DLA  did  not  conduct  higher  level  testing  for  its  remaining  1 5 
mission  critical  logistics  systems  is  correct,  however,  12  of  these  systems  do  not 
require  higher  level  testing  because  they  do  not  exchange  date  information  This 
information  is  partially  captured  in  Appendix  D  of  the  report  The  three 
additional  systems  that  are  not  identified  in  Appendix  D,  which  do  not  exchange 
dale  information,  are  DAASACP,  DEPRA,  and  DMRS.  The  DAISY  system 
participated  in  the  Environmental  Security  end-to-end  test,  FAS  is  a  standalone 
system,  and  FLIS  was  not  identified  by  the  Commanders-In-Chief  or  Principal 
Staff  Assistants  as  a  critical  logistics  process  requiring  testing  Additionally,  all 
mission  critical  systems  were  tested  by  DLA  at  the  Component  level 


DLA’s  contingency  plan  for  the  ninth  mission  critical  system  (MADS)  was 
exercised  on  September  2, 1999. 

The  draft  audit  report  also  concludes  that  DLA  did  not  document  risk  assessments 
for  its  core  logistics  processes  and  systems  and  contributes  to  the  recommendation 
that  die  DLA  CIO  develop  a  risk  management  plan  that  includes  a  risk  assessment 
and  mitigation  strategy  for  its  core  logistics  processes 
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OCT  2  4  W 0 


DLA  has  conducted  and  documented  a  comprehensive  risk  assessment  of  all  of  its 
business  processes.  This  has  been  an  iterative  assessmeot/reassessment  process 
over  a  two-year  period.  The  documentation  is,  and  has  been,  available  but  the 
requests  for  risk  assessment  documentation  was  confined  and  limited  specifically 
to  capstone  planning- 

RECOMMENDATION  1:  We  recommend  that  the  Chief  Information  Officer, 
Defense  Logistics  Agency  develop  a  risk  management  plan  for  inclusion  in  the 
Deputy  Under  Secretary  of  Defense  (Logistics)  plan  that  includes  a  risk 
assessment  and  mitigation  strategy  for  the  core  logistics  processes,  with  special 
emphasis  on  those  processes  and  systems  that  were  not  included  in  higher  level 
testing. 

DLA  COMMENTS:  Concur 

DLA  is  working  with  the  Logistics  Capstone  operational  test  coordinator  who  has 
been  tasked  to  complete  the  DUSD  (L)  risk  assessment  plan.  The  mitigation 
actions  that  result  from  the  DUSD  (L)  risk  assessment  will  be  worked  within  the 
Logistics  Interface  Assessment  Working  Group.  DLA  is  an  integral  part  of  this 
group  and  will  ensure  that  its  core  logistics  processes  and  systems  are  included  in 
the  risk  mitigation  strategy  This  course  of  action  will  also  ensure  a  collaborative 
effort  with  the  Services  for  all  processes  which  cross  Components  boundaries. 

DISPOSITION:  Action  is  Ongoing.  ECD:  November  1, 1999 

ACTION  OFFICER:  Clarence  McNeill,  CIC,  767-2 181 

REVIEW/APPROVAL:  Sandra  King,  Cl 

COORDINATION:  Peggy  Hayes,  DDAI 
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Audit  Team  Members 


The  Readiness  and  Logistics  Support  Directorate,  Office  of  the  Assistant 
Inspector  General  for  Auditing,  DoD,  prepared  this  report.  Personnel  of  the 
Office  of  the  Inspector  General,  DoD,  who  contributed  to  the  report  are  listed 
below. 

Shelton  R.  Young 
Tilghman  A.  Schraden 
Kathryn  L.  Palmer 
Debra  E.  Alford 
Robert  M.  Paluck 
Kayode  O.  Bamgbade 
Mary  K.  Reynolds 


INTERNET  DOCUMENT  INFORMATION  FORM 


A  .  Report  Title:  Defense  Logistics  Agency  Logistics  Year  2000  End-To 
End  Test  Planning 
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